Technical Field
This invention relates to robotic control, and more particularly to a system and method for permitting human manipulation of virtual objects in a randomly varying environment to generate inputs by which a robot interacts with corresponding real world objects autonomously.
Background Information
Throughout this application, various publications, patents and published patent applications are referred to by an identifying citation. The disclosures of the publications, patents and published patent applications referenced in this application are hereby incorporated by reference into the present disclosure.
The field of robotic control includes two distinct classes of control: automated or artificial intelligence control; and remote or distance control. Artificial intelligence (AI), or automated control, relies on a set of preprogrammed responses to stimuli. The robot processes inputs to determine which if any of the pre-programmed responses apply. The system is limited to the quality of the input sources, the degree of pre-programmed responses and variation from the “example material” provided at programming time, and the actual material interacted with by the robot. This AI robotic control tends to be good at doing repetitive tasks, where the robot is configured to do the same thing over and over again. An example of this would be an assembly line, where the robot arm picks up a part and places it in a specific location on a circuit board repeatedly. Robotic devices can also learn to operate in an environment, e.g., by learning how to maneuver around its confines. The ROOMBA® vacuum is an example of such a device. These AI robotic control systems tend to be poor at adapting to uncertain, unfamiliar or changing environments. These control systems generally require prehandling of potential environmental inputs, and failure modes when an input is encountered that was not prepared for can range from ignoring that input to shutting down.
Remote or distance control uses a robotic control system that is physically remote from the robot. A human operator selects inputs to the control system and the robot reacts directly to those inputs, generally in real time. Examples include drone airplanes, underwater remote submersibles, and bomb interacting robots. These robots either fully rely on the human controller to perform control actions or revert to a simple autonomous control in the face of lost input, for example flying in a circle in the case of a drone airplane.
Virtual reality, virtual presentation and virtual worlds may provide representations of the real world, or a simulated world for users to view or interact with. For example, a virtual world may be created which represents real world spaces and attributes, through which a user may navigate.
Robotic control systems may use aspects of virtual reality, such as by overlaying computer generated or computer enhanced graphics onto images of real-world objects to aid in remotely manipulating the real-world objects. For example, a graphical user interface (GUI) may be laid over live images to fuse graphical objects with the physical objects. The resulting augmented objects may be displayed and then manipulated by the user using robotic controls. However, this approach still relies on a human operator to directly control the robot, e.g., in real time.
A need exists for an improved robotic control system that overcomes drawbacks of conventional approaches.